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MISSION STATEMENT 



The mission of the WiBconsin ReBuarch and Devc?lopment Center 
is to ijnprove the quality of education by addressing the full 
range of issues and problems related to individualized schooling. 
Teaching, learning, and the problems of individualiMtion are 
given concurrent attention in the Center's efforts to discover 
processes and develop strategies and materials for use in the 
schools. The Center pursues its, mission by 

= 0 conducting and synthesizing research to clarify the 
processes of school--age children's learning and 
deye lopmen t 

# conducting and synthesizing research to clarify effective 
approaches to teaching students basic ^ skills and concepts 

m developing and demonstrating improved instructional strategies, 
processes, and materials for students, teachers, and school 
administrators , . 

# providing assistance to educators which helps transfer the 
outcomes of research and development ^ to improved practice 
in local schools and teacher education institutions 

The Wisconsin Research and Development Center is supportad 
with funds from the National Institute of Education and the 
University of Wisconsin. 
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This study is a coritinuation of two previous studies designed Uo 
ft* V a 1 t e t e elm i qu o b f o r a s s o s s i n cj c h i Id r on * 3 voc abu 1 a r y kn owled go . 
Three paradigms for assessing vocabulary knowledge were evaluated in 
this study: synonyin, synonyin in contexti and cloEe^ The vocabulary 
tests used were a paper-^and--pencil adaptation of the on-line tests 
used in the two previous studies. 

Overall^ siibjeets performed similarly on all three vocabulary 
formats and scores on all three formats correlated significantly with 
comprehension scores. When subjects* scores oh the vocabulary tests 
were stratified by their performance on a standardised reading compre^ 
hension test^ however, format effects did emerge* Whather more pro^ 
nounced format effects would appear if subjects were grouped into 
finer categories of comprehension level is uncertain* Such elaborate 
stratification procedures were not believed to be justified at this 
point. 

Certainly the question of format effects is an important one^ If 
the particular format used to assess vocabulary causes subjects of 
varying comprehension abilities to perform differentially, the obtained 
scores must be interpreted accordingly. It may well be that no one 
paradigm for assessing vocabulary is most effective for all grades 
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and ability groups ot students^ NGvar tho Ig^b , informaUion is neadod 
about specific tosting modoti, ko that: informoci choicGs can bo mado 

i; 

about which of those modGy in moist appropriate for a particular testing 
purposG. 
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Moasurement: of Vocjibulary Knowl^ocIgQ 

The study dQscribod in this rupC3rt is a continuation of two prG- 
vious studiGS in which tcchniquoR for an;^0S53ing vocabulary knowledge 
wero. asBQSBQd an--line, uning a PLATO tGrminal, In the first study 
(VGn«zky, Porry, CnicoiiOi and Pittelman, 1977) three techniques were 
assossedi Belf-facruoning , word matching under brief exposure ^ and 
synonyms. A follow-iip study (Johnson, Pittelman, Schwenker^ and Perry, 
197B) oxajruned the performanco of elementary school children on five 
paradigms for assessing vocabulary knowledge on-^line: synonym, synonym 
in context^ cloze, oral recognition, and self --selection. 

Due to the inaccessibility and high cost of PLATO terminals in 
most schools, the PLATm> mode of assessmant was discontinued after the 
completion of these two studies. The emphasis of the research was 
then changed from on-line diagnosis, to evaluation through the use of 
a group administered, paper -and -pencil test. 

The main purpose of this current study was to evaluate children's 
perfomance on three paradigms for assessing vocabulary knowledge i 
synonyin, synonym in context, and cloze. The vocabulary tests used 
were a paper -and -pencil adaptation of the on-^line tests used in the 
previous study, with modifications based on information from item analyses 
performed in the previous studies. 



f 




A child who demonstrates knowledge of a word on a particular for- 
mat,, is likely to show that he or she knows those same words when 
tested another way* It is ^ therefore^ important to learn which of the 
many formats utilized for testing word knowledge correlate most highly 
with reading comprehension* Specif icaliy^ this study continued to 
examine the vocabulary test formats of synonym^ synonym in context, 
and cloze, in order to analyzQ patterns of vocabulary test scores 
^id to re lata these formats., to scores on a global measure of reading com 
prehension* 

Most teachers value direct instruction of vocabulary words. The 
work of Davis in 1944, Spear itt in 1972, and others^ has shovm that 
knowledge of word menaings is integral to reading comprehension . 

What are words? Words may be referred to as morphemes, free ' - 
morphemes, physical symbols for concepts, word-length units of meaning, 
graphic configurations bordered by space, or simply words* But, 
however labeled, they are inescapably important components of language, 
which in their' wxHttsn forms ^ must be dealt with by readers* 

In a recent position paper on vocabulary development, Block (1976) 
cited Carroll's (1964) definition of conceptsi 

Words often correspond to, or come to stand for, concepts* 

1 ' 

A "jWord can be thought of as a physical symbol; the meaning of 
trie word corresponds to those aspects of the concePjt that are 
shared by memt^ers of the same speech community; hence, a meaning ■ 
of a word is a societally standardized concept. When' we say 
ttiat a- word stands for or names a concept, it is implied that 



w© are referring to the set of attributes that are commDnly 
undttrntaofi to bu aontainQd in thu ucmuupt* StatGd another 
way, the distinction is often made between a concept as a 
personal mental construct {my personal concept of the attri- 
butes of democracy; youp personal concept .of the attributes 
of democracy) and as a public entity (those attributes of 
democracy that are common to both of our concepts of democracy, 
and shared with other members of our speech conmimity) 
(Block, 1976, pp, 43-44) 

There are many ways of "knowing" a word. Humans have listening/ 
speaking, reading,, and writing vocabularies, which do not totally 
overlap. Some words are recognised aurally but not visually or are ^ 
used' in speaking but not in writing. Fluent reading obviously re^ 
quires high-spead recognition of words and their syntactic arrange^ 
ments. ^^ High frequency words, in particular, must be immediately 
identified by the "fluent reader, ^ 

There has been a long history of attempts to form word lists- 
which are important to learners, as evidenced by the more than 3,000 
entries in the Bibliogra phy of Vocab ulary Studies (Dale, 1965). Basic 
word lists, derived from many sources, have been constructed for a 
nurnber of purposes. Word lists have been compiled from such sources 
as the speech of young children school essays and themes^ language of 
bilingual adults, comic book words, computer analyses, award ^winning , 
children- s. literature, college profanities, andiphonic irregularities. 
But most word lists have been constructed according to word freguencie 
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Otto and Chester {1972) have stated their viewp,,^on the major pur- 
pose of a sight word listi 

Most teachers of reading siibscribe to the notion that beginning 
readers ought to be taught to recognize instantly at sight, 
a number of basic , frequently appearing words. The most 
convincing rationale for teaching sight words is that if 
they are well selected they will, because of their high 
frequency in printed materials ^ have high utility at all 
levels of reading development* (p, 435) 

Sight word lists of .frequently occurring words abound, and most 
instructional programs include their own vocabulary lists. This study 
does not purport to establish a priority list of words children need 
to be taught in order to^ become proficient readers. In practice, 
classroom .teachers determine v/hich words to teach and which meanings 

of those words to emphasize. Most often, teachers base these decision 

if 

on the materials they choose^ the word lists they endorse, the in- 
structional apr>roaches they utilize, and the interests of their 
pupils. But, unfortunately, important diagnostic information is not 
available because the best ways of assessing children on i^ord meanings 
are not yet known. An intensive literature review and ERIC search 
failed to locate even one study which attempted to analyze methods 
of testing word knowledge. The current study was undertaken to answer 
questions "about vocabulary assessment. s 
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Method 

Subjects 

A..ljtotal. of 830 students in grades two through seven participated 
in the study (see Table 1) . The students were from two elementary 
schoDls and one middle school in Brown Deer, Wisconsin, The Brown 
Deer conmunlty has a middle and upper -middle class socioeconomic 
population. All subjects w:ere from heterogeneous classrooms. Testing 
was conducted over a 3-day period in mid^January 1978*" The testing ^ 
session for each class lasted approximately 30 minutes, 

S^t-jjnuli 

: ; The vocabulary test^ used was a group administered^ paper--and- 
pencil test designed to eKamine < paradigms for assessing vocabu^ 
lary knowledge: synonym.^ synonym in context, and clozes For each 
paradigm, there were 8 graduated levels of the test with 10 items per 
level. Each level of the test was color-coded for easy identification*^ 
Target words and response words for all tests were derived from the 
PLATO version of the vocabulary test* These words Ijave been based on 
a selection of words appearing both in standardized vocabulary tests 

and in The Living Word Vocabulary (Dale & 0*Rourke, 1976)* 

— ■ — - • - - — — - — t? 

The Synonym Test required that subjects choose^ from among four 
response words, the word closest in:. meaning to a target word* The - 
Synonym in Context Test presented the target word within the context 
of a sentence*' Subjects were to choose, from among four response words, 
the word closest in meaning to the target word,^ The Cloze Test required 




' Table 1 
Subject Population by 
Sdhool and Grade 

Brown Deer ; - ^ , Total 

School ; "\ per 

District' Grade 2 Grade 3 Grade^^^4^ Grade 5 Grade 6 Grade 7 School 



Elementary 
school 1 

Elementary 
school 2 

Middle 
school 

Total per 
grade' 



20 68 44 ^ 43 175 



90 85 45 74 ' . 294 



156 205 361 



110 ' . 153 89 117 " 156 ' 205 830 
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that subjects select, from among four response words, the word that 
best completed a sentence which had a deletion* Target words ware iden 
tical within each test level across all three formats. Figure 1 shows, 
sample items from Level 4 of the vocabulary test for each of the three- 
test formats. (Appendix A contains the stimuli for all Level 4 tests J 
All levels of the reading subtest of the Metropolitan Achievement 
Tests were Form and were administered in accordance with directions 
specified in the Teacher's Directions manuals, 

PrQcedure . ... ■ 

Two tests were administered to all subjects participating in the 
study I one of the three formats; of the; vocabulary test, and the, appro^ 
priate level of the reading subtest of the Metropolitan Achievement 
Tests (see Table 2), Second through fifth grade' subjects received fihe 
reading subtest of the Metropolitan Achievement Tests first,' 

\ 

All testing took place in the schools and was conducted by members 
of the Project staff. Classroom teachers did not assist with the admin 
istration of tests. Second through fifth grade subjects were testefl in 
their classrooms. Sixth grade siibjects were tested in the middle 
school's reading laboratory, a large room which was partitioned into 
three smaller rooms. Seventh grade subjects were tested in three dif- 
ferent areas within the middle school's Instructional Media^^Center 
(IMC) . ' ' ^, ' 

Each class or group of subjects was assigned to one of the three 
vocabulary test formats. Each subject was tested on three consecutive 
levels of the assigned formats a test one grade below the; .subject" s 



Synonym Test (Level 4) 



2, 


gain 

need have put on lose 


3. 


nei^vous ■■' - 
hungry uneasy '-young tired 


Synonym in Context Test CLeval 4) 


2. 


Ann will gain weight if she eats that for luneh every day* 
need have put on lose 


L 


Patty, bit her nails because she was nervous, ' ! 
"hungry . uneasy , young tired 


Cloz3 Test (Level 4) 




Ann *will ' weight from eating too much. 
need have gain lose 


3* 


Patty bitoh^r nails becuase she was 

smart nervous young asleep / 



Figure 1, Sample items from level 4 c^f the vocabulary test*. ' 
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!; Table 2 .. 
Subjects by Grade Levels Format and Level of Vocabulary Test, . 
and Level of the Metropolitan Achievement Tests 

Level of Metro ^ 





Nuinber of 


'Format and levels of 


poll tan Achievement 


Grade level 


" sutajects 


vocabulary tests 


Tests (Form F) 


2 


21 


Synonym (levels 1, 2^ 3) 


Primary II 


2 


43 


Synonym in Context 
(levels 1, 2, 3) 


Primary 11 


2 


41 


CloEe (levels 1, 2, 3) 


Primary 11 


3 


60 


Synonym (levels 2^ 3, 4) 


Elementary 


3 


47 


S y nonyrn in Co n t ^2ix t: 


Elementary 






(levels 2, 3, 4} 






44 


Cloze (levels 2, 3, 4) 


Elementary . 


.4 


22 


Synonyii^ (levels 3^ 4? 5) 


Elementary 


4 


22 - 


\ Synonyni in Context 
(levels 3, 4, 5) 


. Elementary 




' 52 


Close (levels 3, 4, 5) 


- Elementary 


5 


47 


Synonym (levels 4, 5, 6) 


.. Intermediate 


5. 


. 45, 


^ Synonym in Context 
(levels 4, 5, 6) 


Intermediate 


5 


18 


Cloze (levels 4, 5, 6) 


Intarmediate 


6 - 


'27 


Synon^mi (levels 5^ 6, 7) ^ 


Intermediate , 


6 


20 


Synonym in Context 
(levels 5, 6, 7) 


Intermediate 


6 , 


56 ■ 


Clo^e (levels 5, 6, 7)^ 


Intermediate 
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Table 2 (continued) 



/ 

Niinber of 

^ / 

Gradp level /subjects 



7 
7 



/ 



44 

5 

156 



Fdrmat and levels of 
vocabulary tests 



Synonym (levels 6„ If 8) 

' f 

Synonym in Context 
(levels 6, 7, 8) 
ClOEe ■ (levels 6, 1 , 8) 



Level of Metro- 
politan AchiGVemont 

Tests (Form F) , 
Advanced 
Advanced 

Advanced 
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grade .level. Overall, subjects were tested on 30 items over 3 grade 
levels of the assigned vocabulary test forniat. 

Prior to beginning the vocabulary test, pupils were instructed 
to look at an item on the chalkboard* The chalkboard item was an 
example of the type of items that would appear on the test within the 
assigned Vocabulary test format. The stimulus word or stimulus sentence 
and the four response words, were read aloud by the eKaminer and pupils 
. together g Pupils were asked to select the correct response word from 
the four choices^ 

Following the chalkboard item, pupils read the directions printed 
in their test booklets and Q±d the two practice items beneath the direct 
tions. After the Project staff member discussed the two practice items, 
pupils were 'giyen 5 minutes to work independently on the first level of 
the vocabulary 'test . The examiner then rephrased the test directions, 
and pupils were told to proceed with the next level of the vocabulary 
test. As with the earlier level, pupils were, allowed 5 minutes for test 
completion. The third level of the vocabulary test was conducted in the 
same way as the two preceeding levels* When this last level of the test 
was completed, the vocabulary test booklets were collected . 

Total testing time for all subjects in grades two through seven, 
on both the vocabulary test and the reading subtest of the Metropolitan 
Achievement Tests, ranged from, 60 to 75 minutes, with the longer times 
being taken by younger stude/its. 
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Results 

\ 

\ 

Table 3 presents suimnary data for each of the thre© vocabulary 
assessment formats tested. It should be noted that the mean scores 
shown for each format were calculated on all eight 'levels of the tests 
across the six grades tested in the study. Each subject received three 
levels of a particular format of the vocabulary test; one just below, 
one at^ and one just above his or her grade level. 

Table 4 shows the correlations between the tiuree voGabulary for- 
mats and the reading subtest of the Metropolitan Achievement Tests using 
Pearson Product moment correlations. 

Examinatipn of Table 4 indicates. no markedly high relationship 
for any particular format, either across or within grades, Cor^ela-- 
tions of each format with the comprehension measure, , however, were sig-- 
nificant at the .001 level in each case, with the exception of the 
synonym in context format for seventh grade subjects. Here the small 
cQll size (n^ ^ 5, jj^ ^ .118) precluded meaningful correlation figures. 
Due to an error in collation of the test booklets, most of the data for 
this cell could not bo used. 

Table 5 shows Pearson correlations formats collapsed across grades 
and test levels. The synonym in context format achieved the highest 
overall correlation with comprehension although all three formats 
correlated signif icantiy (at or above ,001) with the criterion measure * 

Table 6 presents the results of t^^tests which were performed on 
scores under the three vocabulary formats. The synonym/synonym in 
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Table 3 

Raw Score SuiTunary Statistics 
□n Vocabulary Tests 



Range of 30 possible 
formats N K SD S£ items 



Synonym 221 19.98 5.31 .35 2^30 



Synonym in 
Contexu 



182 20,95 5.70 .42 2-30 



Cloze 367 20.88 5.65 ,29 3-30 

Total Group 770 2o\63 5,57 ,20 2-30 
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Table 4 

"Pearson Correlations of Vocabulary 
Formats x Metropolitan Subtest 
Listed by Grade 



Grade 



2 
3 

4 
5 

6 
7 



Synonym 



.7341 
.7746 
.7377 
.6925 
.6455 
,7 396 



Format 
Synonym in Context 



.6982 
.7711 
*6360 
.7287 
.7446 
.6495 



a 



Cloze 



.8088 
,7011 
.6523 
,8584 
. 7914 
,6646 



Cell size ^ 5, £ ^ .118^ all otliers significant at .OOr level, 



or 



Table 5 

Pearson Correlations of Vocabulary 
Formats x Metropolitan Subtest 
Across Grades 

Synonym Synonym in ConteKt Cloae 

,6730 ,6884 ' ,6410 



Table 6 

^--test Comparisons of Vocabulary 
Formats (Unstratif ied on Metropolitan Subtest) 
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I significance 
Comparisons X Difference t^-^value probability 



9 ^ . 



\ 

\ 

\ 
\ 



Synonym/Synonym in 1.38 4,093 ' .00001 

Context . ^ . . 

Synonym/Cloze ^. 73 2.534 .0115 ' 

Cloze/Synonym in -.65 -2;i39 .0327 

Context 



context compai^ison yielded a t^--valua of 4,093, significant at the *00001 
level,' The synonyin/cloze comparison also achieved significance wi,th a 
t-^value of 2,534 at" the ,0115 level. The cloze/synonym in context com^ 
parison fell short of significance at the .01 leve]. with an absolute 
value for t of 2,139 at the .0327 level. 

Table 7 presents the results of a one-way analysis of variance 
which was performed to examine the main effect of format. This is in- 
dicatad on the table under the heading "between groups," The F_-ratio 
for the format effect was 2,190 with a probability of *110. Because 
this F^value may have been lowered somev;hat by the wide variation among 
subjects* reading comprehension abilities within format groups, it was 
decided that subjects' scores should be stratified into three comprehend 
sion level groups based on their, performance on the criterion measure 
(the reading subtest of the Metropolitan Achievement Tests) * Analysis 
^on the stratified groups was then performed to determine whether the 
three comprehension groups perfomed differentially on^the three vocabu-^ 
lary formats, , ^ 

' On the basis of a cumulative frequency table of subjects' raw 
scores on the reading subtest oS^he Metropolitan Achievement Tasts, cut 
off points were established which 'divided subjects into low, medium, or 
high comprehension levels* The comprehension ability groupings which 
resulted from this stratification procedure had the characteristics 
shown in Table 8, 

Table 9 presents smnmary statistics on the nine comprehension 
levels X format subgroupings . 
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Table 7 
One Way Anova of 
Main Effect of Format 



Source DF S£ MS _F=Ratio F--Probability 
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Between 2 135.6758 67.8379 2.190 .110 

groups 

Within 772 23917.9883 30.9818 

groups 
f 

Total 774 24053 •6641 



Table 8 
Summary Statistics of 
Comprehension Ability Groups 



ComprehensiDn Ffircent of Range on Metropoiitan Subtest 

level N_ ^ total ^ (of 44 possible) 

High .2 38 28,44 34-'44 ^ 

Medium 313 40.65 23-33 

Low • 219 28.44 ^ ^ 3-22,^ 



? 
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. Table 9 . 
Sunmiary Statistics of Format Groups 
Stratified . into Compreheasion Ability Groups 



Format 






Range on Metropolitan Subtest 




Vari- 




SE 


Groupjs 




N 


I (of 44 possible) 


X 


ance 


SD 


ofi X ' 




.LOW 


55 


7-22 


17,53 


17,40 


4,17 


.562^ 


Synonym 


Medium 

/ 


89 


23-33 


27,7tf 


10,12 


3.18 


,337 




High 


77 


34-42 


37.55 


' 5.49 


2,34 


,267 


Synonym 


Low 


58 


V 3-22 


16,48 


17,34 


4,16 


,547 


in 


Medium 


72 


23-33 ' 


28.06 


8,45 


2,91 


,343 


Context 


High 


52 


34-44 


38,04 


9,68 


3,11 


:*432 


\ 


Low 


106 


6-22 


16,80 


20*50 


4,53 


,440 , 


Cloze \ 


Medium 


152 


23-^33 


28,53 


9,48 


3,08 


,250 




\'igh 


109 


, 34-43 


37,72 


6.39 


2,53 


, 242 


Total 




770 


3-44 


27,93 


^ 75,72 


8,70 


,314 
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All-ways comparisons ware next made via ^-tests. Results of these 
analyses are swnmarized in Table 10. As was expected, all comparisons 
both within and between formats between high and low comprehension levels^ 
were sighificant at or above the .OOOQl level* 

Table 11 presents the within-f ormat comparisons of low versus 
medium comprehenders and of meditim versus high Gomp^ehenders . Again,, 
all t-values were uniformly high and significant at or above the ,00,001 

level. ' . 

^ The comparisons of real interest, however, were those of same^ 

comprehension level (SCL) iiibjeGts whose^ vocabulary knowledge was assessed 

by different formats. Results of these t^-test compartpons are presented 

in Table 12. Two comparisons did show interesting differences, although 

no t-values were significant at the .05 level. Low comprehenders who 

took the synonyTn format performed somewhat better (mean difference ^ 1*04) 

than low comprehenders who took the synpniro in context format. The t;- 

value for this comparison was 1,65 at a probability le^el of .0993. 

The other comparison of note was that between medium comprehenders taking 

the synonym format with medium comprehenders taking the cloze format 

of the vocabulary test. The mean difference between these two SCL 

groups was 0,83 witii a t-valua of 1,85 with a probability of *0649. 

All other SCL comparisons had t-values of 1.3 or less and significance 

levels of 0.195 or more. 

The final analysis of the data consisted of sequential tests via 
two'-way analysis of variance under first Fixed and then 'Random Effects 
models.^ Table 13 presents the results of these tests* 
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TablD 10 ■ ' 


/ 






t-tcsts 


of Unweighted Means of 
Vocabulary 


Factors of the MQtropolitan Subtest and J 
on Stratified Groups ' 






\.QW 




mm 




/Ldw CDntt'?£t/Hf-*Hium Cnntf-'St/Hiqh 


Clozc/ 
Modi tin 








































• 






































B>.OQ01 
















t^igji 




- 






















P>,00D1 
























p. 0001 , 


£^.0001 




£>.nr)ni 




5 




>1,30 














P>.1546 


D>,0001 






' £>,Qooi p>.oo6i ■ 






t=20,79 




pi 9 .18 






■ ; 




p>.oaoi 




£>,00C1 




1^.3281 p>.Q001 , £>.0Q01 










t= ,358 




t-18.94 t= .554 Ml, 62 










p=,?203 


£>,aooi 




£>.000l " 
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Tabla 11 

Comparison of Comprehension Groups 
Within Formats of Vocabulary Measure 



Forntats 



Synonym ■ , 

Synonym in Context 
CloEe 



Low 1. 1 Medium 



t^l7,63 
t^l9,51 
t^27.55 



Medium i i High 



t^l8.S2 
tfl6.31 
t^21,80 



Note. All t-values significant at or above, ,00001 level. 
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Table 12 

C oni p a r i b on of C otTi p i: g h e n s i on Ah i 1 i t y 
Groupings Acroas Vocabulary Formats 



Formats Low Medium High 



Synonyin /Synonym 

t_^1.65 (.0993) t^,573 (,5009) t^^.817 (.4143) 
^ in Context ^- ' " 



Synonym/cloze t^l.30 (,1946) t^l.85 (,0649) (,7203) 

Synonym in Context/ 

t^^,581 (,5614) t^^979 U3281) ^ t^»554 (,5801) 
Cloze . " 



Note ~ Significance levols shown in parentheses* 
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Source 



Tabla 13 

Sequential Hypotheses Test Via Two Way Anova 
Under Fixed and Random Effects Models' 



df 



Dependent 



variable 



F -ratio 



Fixed Effects M^el 



Probability 
level 



Format 

Metropolitan 
= Subtest 

Ihteraction 



761 

2 
761 

■4 

761 



Metropolitan 
Vocabulary 

Metropolitan 
Vocabulary 

Metropolitan 
Vocabulary 



^ ,3842 
6.3831 

2191,13 
222,96 

1.5869 
.3567 



.6811 
.0018 

.0000 

.0000 

.1758 
,8394 



Random Effects Model 



Format 

Metropolitan 
Subtest 

Interaction 



2 
4 



761 
761 



Metropolitan 
Vocabulary 

Metropolitan 
Vocabulary 

Metropolitan 
Vocabulary 



.2421 

17,8933' 

2191,13 
222.960 

1.5869 
,3567 



.7957 
,0101 

.0000 
,0000 

,1758 
.8394 
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The format effecti on the vocabulary measures was significant at 
the .01 level with F^ratios of 6.38 and 17.89 under the Fixed and Random 
Effects models^ rospoctivcly , No in tor act ion effects wore present in 
either case. 

Subjects porforTnod similarly on the three assessment formats across 
the six grades tested. No one test format perforiTied remarkably better 
than either of the others within or across grades. But^ there were sig^ 
nif leant mean differences in scores obtained under the three formats 
overall. These differences indicated synonym^ synonym in context, and 
cloze in rank order of difficultyf from eas j "ist to most difficult. 

When subjects were stratified by comprehension ability^ once 
again scores on any one format were not significantly higher overall/ 
than scores on either of the other two formatn. Yet/ some differences 
did appear. Low comprehenders scared slightly higher on synon>an than 
on synonym in context^ and medium comprehenders scored somewhat higher 
on cloze than on synonym in context* 

These findings are in contrast to the results presented in the PLATO 
IV report (Johnson et al,^ 1978). In the studies described in that 
report, on-line testing of the same three vocabulary formats was con-^ 
ducted, and the reading subtest of -the Metropolitan AchievOTent Tests 
was also administered* In these studies, however, fourth and fifth 
grade scores showed far higher con ^lat ions between vocabulary and 
comprehension scores than did second and third grade scores* The over^ 
all resiilts were similar in one important respects no one fonnat of 
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vocabulary assessinent showed consistently higher corrGlations across the 
gradQ levels tasted. 

It is possible that further differencGS in vocabulary performancQ 
might appear if subjects were grouped into finer categories of compre- 
hension level. Such elaborate stratifications were not believed to be 
justified at this point, beGause investigators questioned the basis for 
making fine discriminations among perf ormances based on a single stan- 
dardized measure of comprehension i But, future studies of format effects 
in vocabulary assessment might involve other, concurrent measures of 
comprehension ability. Ultimately, the validity and nature of, the 
format effects discovered in the study described above rest on the 
validity and reliability of the criterion measure of comprehension used. 
As more effective comprehension assessment instruments become available, 
more confidence can be placed in the results of studies on format effects 
which utilize those comprehension assessment instruments as criterion 
measures* - ' 

Although the findings of this study did not demonstrate the 
superiority of one vocabulary paradigm over another, some interesting 
format effects did appear. Further examination of the question of for- 
mat effects on the assessment of vocabulary knowledge is certainly 
warranted. If a particular format used to assess vocabulary knowledge 
yields lower scores than another format , thi's^'fiformation should be 
made available to educators 'Utilizing those scores* It may well bev 
that no one paradigm for assessing vocabulary is most effective across 
all grades and ability groups of students. Nevertheless, information 



2H ^ 

is needed about specific: aaBQUsmGnt modG£>, bo that informud choices can 
iiial^^ about which ot tlioaQ modcy aro appropriate for particular to sting 
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Appendix A 

Stimuli from Level 4 of the Vocabulary Tests 



Synonym trtstj Level 4 

Synonym in Contaxt Test, Level 4 

CloEa Test, Level 4 
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Synonyni Test 



Level 4 



33 



Stimulus 

gigantic 

gain 

nervous 

broken 

concealed 

feeble 

briskly ' 

exhibits 

surprised 

twisted 



Responses 



huge 

need 

hungry 

destroyed 

wanted 

funny 

slowly 

acts 

sick 

held 



baby 



have 

uneasy 

shaken 

discovered 

unhap^^^ 

quickly 

people 

gone 

woiind 



friendly 

put on 

young 

dirtied 

found 

weak 

sadly 

rides 

bored 

found 



lonely 

lose 

tired 

moved 

hid , 

strong 

awkwardly 

displays 

ainazed 

placed 
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Synonym 1n Context Test 



Level 4 



Stimulus 

(The target word is 
underlined* ) 

"^^^ gigantic giraffe licked the child's 
hand. 



Responses 



huge 

friendly 



lonely 



Ann^ will gain weight if she eats that 
for lunch everyday . 



need 
put on 



have 

lose 



Patty bit her nails because she was 
nervous* 



hungry 
young 



uneasy 
tired 



Many of our dishes were broken by the 
earthquake . 



destroyed shaken 
dirtied moved 



He concealed the gun in his coat pocket* 



wanted 
found 



discovered 
hid 



The feeble old dog walked beside tha 
woman * 



funny 
weak 



unhappy 
strong 



The horse trotted briskly along the path. 



slowly 
sadly 



quickly 
awkwardly 



There were many interesting exhibits at 
the fair. 



acts 
rides 



people 
displays 
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Stimulus 

(The target word is 
underlined*) 

Mark was surprised when his friend from 
Texas arrived. 



Responsfs^ 



sick 
bored 



gone 
amazed 



Cathy twisted a rubber band around her 
finger* 



held 
found 



wound 
placed 



EKLC 



•I 
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. - : Cloze Test 

Level 4 ^ 

stimulus ■ Responses 

The giraffe was so that he made gigantic young 

even the elephants look small. . ^ 

friendly lonely 

Ann will _ _ weight from eating too need hav.e 

much. 

■ . gain lose 

Patty bit her nails because she was smart nervous 



young asleep 

Mdny of our dishes were / by the broken shaken 
earthquake and we had to sweep up all 

the pieces. lost moved ' 

He the gun in his coat pocket so made discovered 

no one would know he had it. 

found concealed 

The ' old dog could barely keep up quick unhappy 
with the woman, but wagged his tail 

' happily anyway. ^ feeble stroncf 

The horse trotted along the path slowly briskly 
; and -arrived home sooner than usual. 

j sadly awkwardly 

There Were many interesting at ^yjt^^ats people 

the science fair to show how different . / 

foods are grown. / rides ^ exhibits 

'■ " - / 



stimulus 

Mark was and happy when His 

friend arrived uneKpectedly from 
Texas. 



Responses 
sick gone 
bored surprised 



Cathy 



around so she could reach 



the book on the shelf behind her. 



held 
found 



twisted 
placed 
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